Skip to content

feat(agent): view every row of the data in a read-only grid - #12513

Merged
MarkusNeusinger merged 8 commits into
mainfrom
feat/agent-data-grid
Nov 2, 2026
Merged

MarkusNeusinger merged 8 commits into
mainfrom
feat/agent-data-grid

Conversation

@MarkusNeusinger

@MarkusNeusinger MarkusNeusinger commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • .view_all() next to the agent chat's data preview opens the whole current data version in a read-only @mui/x-data-grid Community grid (MIT), loaded on demand. You get a search over every column, sorting, one column filter, column resizing and hiding, and the dtype under each name, in both site themes and at phone width.
  • A new read-only route, GET /debug/agent/sessions/{sid}/dataset/data.csv, serves the canonical CSV as stored, and a strict RFC 4180 reader parses it in the browser. Every cell stays the file's text, so nothing is rounded or reformatted.
  • Opening the view reports the content-free outcome opened_data_view (version 0: the session's data, not a result).

Plan

Owner decisions of 2026-11-01. No plan file.

Decisions and limits

The owner chose pages of 25, 50 or 100 rows, default 100, because the MIT grid forces pagination. Search, filter, and sort run over all rows, and the footer reads 1–100 of 10,000.

Limits of the MIT edition, accepted for now:

  • One column filter at a time.
  • One sort column at a time.
  • No column reorder.
  • At most 100 rows a page.

Routine decisions made along the way:

  • Empty cells sort last in both directions, for every column. The grid negates a comparator for desc, which would open a descending sort on a page of gaps.
  • The grid's panels and menus mount inside the dialog. Mounted in the body, the filter panel stayed aria-hidden once a select inside it closed, because MUI's modal manager hides every sibling of the top modal.
  • The page size stays available at phone width. The grid hides the picker below sm.
  • A chunk that does not load shows a line in the card (lazyOrFallback). The route's error boundary would reload the page, and a reload ends the session, whose state lives only in the page.
  • dataView.ts became viewTable.ts, and PreviewCell.tsx became CellText.tsx. Each differed from a sibling only by case, which resolves to the wrong file on a case-insensitive checkout.

Open points

  • TanStack Table stays an open option if the MIT limits start to matter.
  • Turning a view filter into a data step is out of scope.

Deploy order

The page handles either order. Without the new agents route the view says this server cannot show every row yet (404 not_found), and the page ignores a refused outcome beacon.

Numbers

Bundle, against origin/main with VITE_ENABLE_AGENT_CHAT=true:

Chunk Change
New lazy DataView chunk 479.6 KB, 140.8 KB gzip
mui (loads on every page) +2.0 KB, +0.7 KB gzip
index +22 B

With the flag off, which is the default build, only agent.js changes (+83 B). The grid is in no always-loaded chunk.

Timings on 10,000 rows by 30 columns (1.69 MiB), production build of 1691a36d6, headless Chromium, from click to painted:

Action Time
Open the view 816 ms
Page change 54 ms median, 373 ms for the first
Quick filter 306 ms, of which 250 ms is the debounce
Sort 77 ms
Column filter 166 ms

Test plan

  • yarn lint, yarn fm:check, yarn type-check, yarn test (1,423 tests), and yarn build.
  • ruff check, ruff format --check, mypy api core, and pytest for the touched API and agents tests (424 plus 75 for telemetry).
  • Headless Chromium against app/scripts/agent-bff-mock.mjs on a production build of 1691a36d6, the head with Copilot's fixes merged with main, 96 of 96 checks passed: desktop light and dark, 390 px light and dark, and 320 px. Search, the numeric sort in both directions, and a > filter are compared with results computed independently from the CSV. Also checked: no horizontal page scroll, Escape's order (dropdown, panel, view), and focus back on the button after closing.
  • Column menu, columns panel, page-size change, and a phone in landscape.
  • A blocked DataView chunk: the line in the card, no reload, the data stays, focus returns.
  • csv.ts against Python's csv.writer in the service's dialect: 4,008 generated tables with hostile cells, all identical.
  • /audit-licenses: no new binaries or data URIs; all eight new lock entries are MIT; no -pro, -premium, or x-license.
  • /zweitmeinung with Gemini 3.8 Flash: four findings. Applied: text order in a descending numeric sort, and the version in the request identity (hardening). Rejected after a browser run: Escape in a filter dropdown closes only the dropdown. Skipped: the Vite memo under vite build --watch, which nothing runs.
  • Copilot's one round: two low findings, both applied. A bare carriage return outside a quoted cell gets its own message, and a test title no longer overstates its fixture.
  • Owner check on prod after the merge: open .view_all() once on real data. The real BFF and agents path was only driven through the mock, never end to end. The agents route and the BFF route each have their own tests, and MAX_DATASET_CSV_BYTES is pinned equal to MAX_CANONICAL_BYTES.

https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8

MarkusNeusinger and others added 5 commits November 2, 2026 00:34
The data panel's `.view_all()` opens the current data version in the MIT
MUI X Data Grid, loaded on demand: search over every column, sorting
(numeric columns by number), a column filter with `>`/`<` on numbers,
column resize and hide, the dtype in each header, both themes and phone
width. A new read-only route serves the canonical data.csv, which a strict
RFC 4180 reader parses in the browser, so every value shows exactly as
stored. Opening it reports the outcome `opened_data_view`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
A review of the first commit, with a headless run against the BFF mock on
10,000 rows by 30 columns, found what the unit tests could not see:

- A descending sort opened on a page of blanks: the grid negates a column's
  comparator for `desc`, which turned "empty cells last" into "empty cells
  first" (103 gaps in the test column, a page holds 100). Each column now
  has a comparator per direction, so the gaps stay last either way up, for
  text columns too.
- The filter panel vanished for screen readers after the first column
  choice: mounted in the body, it sits beside the modal dialog, and MUI's
  modal manager hides every sibling of the top modal once a select inside
  closes. The grid's panels and menus now mount inside the dialog, capped
  to its room on a short screen.
- The page size could not be changed at phone width: the grid drops the
  picker below `sm`. It shows now, without its label.
- `dataView.ts` beside `DataView.tsx` and `previewCell.ts` beside
  `PreviewCell.tsx` differ only by case, and on a case-insensitive checkout
  the first import resolves to the wrong file. They are `viewTable.ts` and
  `CellText.tsx` now; main had no such pair.

A chunk that does not load (a deploy replaced its hash, the connection
dropped) reached the route's error page, which reloads and so ends the
session, whose state lives only in the page. `lazyOrFallback` keeps the page
and says so in the card; a failed preload stays quiet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
Text in a numeric column now reverses with the sort and stays ahead of the
empty cells, through the same rule as a text column. A view's request names
the version as well as the profile, so the code does what its comment says;
every version arrives as new JSON today, so this hardens rather than fixes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
Copilot AI lite review requested due to automatic review settings November 2, 2026 00:22
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two unresolved findings remain: the CSV parser reports a bare carriage return in an unquoted cell as a 'text after a quoted cell' error, and a test title overstates its fixture as the largest storable table when it exceeds the 4 MiB cap.

2 open issues
What changed in this PR

This PR adds a read-only "view every row" grid to the agent chat's data card. The .view_all() button opens every row of the current data version in an MUI X Data Grid (MIT edition), loaded only on demand. Users can search all columns, sort, apply one column filter, resize and hide columns, and see each column's dtype. The grid reads the stored canonical CSV through a new read-only route, so values appear exactly as stored. Opening the view also sends a content-free opened_data_view outcome.

Changes:

  • A new read-only API route serves the current dataset's data.csv, through both the BFF and the agents service, capped at 4 MiB and gzip-compressed in a worker thread.
  • A strict RFC 4180 reader parses the CSV in the browser, and the grid is built from that table. A new lazy chunk holds the grid, and a Vite rule keeps it out of the always-loaded mui chunk.
  • The view handles phone widths and both themes. If its chunk fails to load, the card shows a line instead of reloading the page.
File Description
tests/​unit/​api/​test_agent_router.py Tests the BFF CSV route: headers, gzip in a thread, size cap, and upstream error codes.
tests/​unit/​agents/​test_artifact_names.py Pins the BFF CSV cap equal to the canonical-CSV cap in the agents service.
tests/​unit/​agents/​runtime/​test_dataset_csv_route.py Tests the agents service CSV route: stored bytes, new versions, errors, and a turn in flight.
docs/​reference/​api.md Documents the new CSV route and the opened_data_view outcome.
docs/​reference/​agent-telemetry.md Adds opened_data_view to the outcome telemetry line.
docs/​concepts/​agent-network.md Adds the CSV route to the agent network route table.
changelog.d/​agent-data-grid.md Changelog fragment for the new view.
app/​yarn.lock Locks the MUI X Data Grid and its new MIT dependencies.
app/​vitest.config.ts Makes design tokens readable in tests so a theme test can compare them.
app/​vite.config.ts Keeps the grid's modules out of the always-loaded mui chunk.
app/​src/​sections/​agent-chat/​viewTable.ts Builds the table from the CSV, with dtypes and column widths, and the number-aware sort.
app/​src/​sections/​agent-chat/​viewTable.test.ts Tests table building, widths, number parsing, and sort order.
app/​src/​sections/​agent-chat/​messages.ts Adds the user-facing failure lines for the full view.
app/​src/​sections/​agent-chat/​messages.test.ts Tests the full view's failure lines.
app/​src/​sections/​agent-chat/​lazyOrFallback.ts Lazy-loads a component and renders a fallback if the chunk fails.
app/​src/​sections/​agent-chat/​lazyOrFallback.test.tsx Tests the lazy loader's success and fallback paths.
app/​src/​sections/​agent-chat/​dataViewTheme.ts Builds the grid's MUI theme in light and dark, using the site's colour tokens.
app/​src/​sections/​agent-chat/​dataViewTheme.test.ts Checks the theme's colours match the site's CSS tokens.
app/​src/​sections/​agent-chat/​dataViewGrid.tsx Builds grid columns and rows, including numeric comparison filters.
app/​src/​sections/​agent-chat/​dataViewGrid.test.tsx Tests grid rows, columns, sorting, and numeric filters.
app/​src/​sections/​agent-chat/​DataView.tsx The full-view dialog: search, toolbar, loading, retry, and Escape handling.
app/​src/​sections/​agent-chat/​DataView.test.tsx Tests the dialog's display, retry, close, search, and new-version reload.
app/​src/​sections/​agent-chat/​DataPanel.tsx Adds the .view_all() button and mounts the lazy view.
app/​src/​sections/​agent-chat/​DataPanel.test.tsx Tests the button, focus return, read-only opening, and the fallback line.
app/​src/​sections/​agent-chat/​csv.ts Strict CSV reader that keeps every cell as its exact text.
app/​src/​sections/​agent-chat/​csv.test.ts Tests the CSV reader against the service's output and malformed input.
app/​src/​sections/​agent-chat/​CellText.tsx Shared cell renderer, extracted from the preview, for line breaks and space runs.
app/​src/​pages/​AgentChatPage.tsx Wires the data loader and the opened_data_view outcome into the data panel.
app/​src/​lib/​agent.ts Adds the datasetCsv client call and the new outcome action.
app/​src/​lib/​agent.test.ts Tests the datasetCsv client call and its errors.
app/​src/​hooks/​useAgentSession.ts Exposes a session-scoped fetchDataCsv to the page.
app/​scripts/​agent-bff-mock.mjs Mock BFF: serves the CSV route and generates large tables for manual testing.
app/​package.json Adds the @mui/x-data-grid dependency.
api/​routers/​agent.py BFF route for the CSV with a size cap and gzip handling; adds the outcome action.
agents/​main.py Agents service route returning the stored canonical CSV.
agents/​anyplot/​telemetry.py Adds opened_data_view to the allowed outcome actions.

Review effort: Lite

Comment thread app/src/sections/agent-chat/csv.test.ts Resolved
Comment thread app/src/sections/agent-chat/csv.ts Resolved
@codecov

codecov Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.37500% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
app/src/pages/AgentChatPage.tsx 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

MarkusNeusinger and others added 2 commits November 2, 2026 06:50
A bare carriage return outside a quoted cell now gets its own message
instead of "text after a quoted cell", and a test no longer claims that a
fixture of about twice the service's cap is the largest table it stores.
New tests cover `fetchDataCsv` and the view's line when no answer comes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
@MarkusNeusinger
MarkusNeusinger merged commit 431656e into main Nov 2, 2026
16 checks passed
@MarkusNeusinger
MarkusNeusinger deleted the feat/agent-data-grid branch November 2, 2026 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants